Camera device and  image processing method

ABSTRACT

The invention provides a camera device. In one embodiment, the camera device comprises a sensor and a controller. The sensor detects an image to generate a first image signal with an RGB format. The controller comprises an image processor and a subsequent processor. The image processor converts the first image signal to a second image signal with a YUY2 format. The subsequent processor adjusts a plurality of luma components, a plurality of first chroma components, and a plurality of second chroma components of the second image signal to obtain a plurality of adjusted luma components, a plurality of first adjusted chroma components, and a plurality of second adjusted chroma components of a third image signal. A host receives the third image signal output by the camera, and uses a Direct Show module to convert the third image signal to a fourth image signal with an RGB format.

This Application claims priority of Taiwan Patent Application No.98141474, filed on Dec. 4, 2009, the entirety of which is incorporatedby reference herein.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention relates to camera devices, and more particularly to imageprocessing of camera devices.

2. Description of the Related Art

A Direct Show program is an application programming interface producedby Microsoft for software developers to perform various operations withmedia files or streams. A Windows system developer can use the DirectShow program to play or record media files. Ordinarily, a Direct Showprogram is often used to receive video data generated by a camera andshow the video data on a screen coupled to a host. In addition, theDirect Show program is also a component program of a Windows MediaPlayer for playing multi-media files.

A host executing a Direct Show program receives video data of a varietyof formats from a camera. Ordinarily, a camera device can output videodata of a YUV format or a motion joint photographic experts group (MJPG)format. According to the YUV format, a pixel of an image is divided intoa luma component and two chroma components. The YUV format is furtherdivided into Y′UV, YUV, YCbCr, YPbPr, and YUY2 formats, the differencebetween which is the numbers of luma components and chroma componentsfor representing a pixel. According to the MJPG format, pixels of animage are compressed and encoded to reduce the data amount of the image.The host must therefore decode image data of the MJPG format before thehost shows the image data of the MJPG format.

When the host shows the image data with the MJPG format on a screen, theluminance and chrominance of the image data shown on the screen differfrom those of an image of a YUY2 format. A screen generally can onlyreceive image data with a RGB format. A host therefore must convertimage data to the RGB format before the image data is shown on thescreen. The Direct Show program provided by Microsoft, however, convertsimage data of a YUY2 format and an MJPG format to image data of a RGBformat with different transform coefficients, leading to difference ofluminance and chrominance of the image data converted from the YUY2format and the MJPG format. To make image data converted from the YUY2format and the MJPG format have the same luminance and chrominance, acamera device capable of adjusting the luminance and chrominance ofoutput image data is therefore required.

BRIEF SUMMARY OF THE INVENTION

The invention provides a camera device. In one embodiment, the cameradevice is coupled to a host, and comprises a sensor and a controller.The sensor detects an image to generate a first image signal with an RGBformat. The controller comprises an image processor and a subsequentprocessor. The image processor converts the first image signal to asecond image signal with a YUY2 format. The subsequent processor adjustsa plurality of luma components, a plurality of first chroma components,and a plurality of second chroma components of the second image signalto obtain a plurality of adjusted luma components, a plurality of firstadjusted chroma components, and a plurality of second adjusted chromacomponents of a third image signal. The host receives the third imagesignal output by the camera, uses a Direct Show module to convert thethird image signal to a fourth image signal with an RGB format, andoutputs the fourth image signal to a screen.

The invention provides an image processing method. In one embodiment, acamera device is coupled to a host. First, an image is detected by asensor to generate a first image signal with an RGB format. The firstimage signal is then converted by an image processor to a second imagesignal with a YUY2 format. A plurality of luma components, a pluralityof first chroma components, and a plurality of second chroma componentsof the second image signal are then adjusted by a subsequent processorto obtain a plurality of adjusted luma components, a plurality of firstadjusted chroma components, and a plurality of second adjusted chromacomponents of a third image signal. The third image signal is thentransmitted to the host. The host receives the third image signal outputby the camera, uses a Direct Show module to convert the third imagesignal to a fourth image signal with an RGB format, and outputs thefourth image signal to a screen.

A detailed description is given in the following embodiments withreference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention can be more fully understood by reading the subsequentdetailed description and examples with references made to theaccompanying drawings, wherein:

FIG. 1 is a block diagram of a camera device generating an image signalof an MJPG format and a host receiving the image signal;

FIG. 2 is a block diagram of a camera device generating an image signalof a YUY2 format and a host receiving the image signal;

FIG. 3 is a schematic diagram of a frame of an image signal of a YUY2format;

FIG. 4 is a block diagram of a camera device outputting an image signalof a YUY2 format and a host receiving the image signal according to theinvention; and

FIG. 5 is a flowchart of a method for processing image signals of thecamera device and the host shown in FIG. 4 according to the invention.

DETAILED DESCRIPTION OF THE INVENTION

The following description is of the best-contemplated mode of carryingout the invention. This description is made for the purpose ofillustrating the general principles of the invention and should not betaken in a limiting sense. The scope of the invention is best determinedby reference to the appended claims.

Referring to FIG. 1, a block diagram of a camera device 102 generatingan image signal of an MJPG format and a host 104 receiving the imagesignal is shown. The camera device 102 generates an image signalS_(MJPG) of an MJPG format. In one embodiment, the camera device 102comprises a sensor 112 and a controller 114. The sensor 112 detects animage to generate an image signal S_(RGB) of a RGB format. Thecontroller 114 comprises an image processor 116 and an MJPG encoder 118.The image processor 116 receives the image signal S_(RGB) generated bythe sensor 112, and converts the image signal S_(RGB) to an image signalS_(YUV) of a YUV format. The MJPG encoder 118 then receives the imagesignal S_(YUV) generated by the image processor 116, and then compressesthe image signal S_(YUV) to obtain an image signal S_(MJPG) of an MJPGformat.

The camera device 102 then transmits the image signal S_(MJPG) to a host104. In one embodiment, an universal serial bus (USB) is coupled betweenthe host 104 and the camera device 102 for data transmission. Theuniversal serial bus is driven by a USB video class driver 122 of thehost 104, and the image signal S_(MJPG) is sent to the host 104 via theuniversal serial bus. The host 104 has a Windows operation system whichcomprises a Direct Show module 124 for playing the image signalS_(MJPG). In one embodiment, the Direct Show module 124 comprises anMJPG decoder 126 and a color converter 128. The MJPG decoder 126 decodesthe image signal S_(MJPG) of an MJPG format to generate an image signalS_(YUV) of a YUV format. The color converter 128 then converts the imagesignal S_(YUV) to an image signal S_(RGB) of a RGB format. Finally, thehost 104 sends the image signal S_(RGB) to a screen 106, and the screen106 displays the image signal S_(RGB).

Referring to FIG. 2, a block diagram of a camera device 202 generatingan image signal of a YUY2 format and a host 204 receiving the imagesignal is shown. The camera device 202 generates an image signalS_(YUY2) of a YUY2 format. In one embodiment, the camera device 202comprises a sensor 212 and a controller 214. The sensor 212 detects animage to generate an image signal S_(RGB) of a RGB format. Thecontroller 214 comprises an image processor 216. The image processor 216receives the image signal S_(RGB) generated by the sensor 212 andconverts the image signal S_(RGB) to an image signal S_(YUY)2 of a YUY2format. The camera device 202 then sends the image signal S_(YUY)2 tothe host 204.

Referring to FIG. 3, a schematic diagram of a frame 300 of an imagesignal of a YUY2 format is shown. A pixel of the frame 300 of the YUY2format is represented by a luma component Y, a first chroma component U,and a second chroma component V. The luma component Y indicatesluminance of the pixel, and the chrominance of the pixel is indicated byboth the first chroma component U and the second chroma component V. Ifall pixels of the frame 300 are represented by the luma component Y, thefirst chroma component U, and the second chroma component V, the totaldata amount of the frame 300 is massive. To reduce the total data amountof the frame 300, each pixel of the frame 300 of a YUY2 format comprisesa luma component Y and a chroma component selected from U and V. Forexample, a first pixel of the frame 300 comprises a luma component Y0and a first chroma component U0, and a second pixel of the frame 300comprises a luma component Y1 and a second chroma component V0. When aframe 300 of a YUY2 format is displayed, the first chroma component Uand the second chroma component V must be calculated according to aninterpolation method.

In one embodiment, an universal serial bus (USB) is coupled between thehost 204 and the camera device 202 for data transmission. The universalserial bus is driven by a USB video class driver 222 of the host 204,and the image signal S_(YUY2) is sent to the host 204 via the universalserial bus. The host 204 has a Windows operation system which comprisesa Direct Show module 224 for playing the image signal S_(YUY2). In oneembodiment, the Direct Show module 224 comprises a color converter 228which converts the image signal S_(YUY2) to an image signal S_(RGB) of aRGB format. Finally, the host 204 sends the image signal S_(RGB) to ascreen 206, and the screen 206 displays the image signal S_(RGB).

When the host 104 shown in FIG. 1 and the host 204 shown in FIG. 2respectively display the image signal S_(MJPG) of an MJPG format and theimage signal S_(YUY2) of a YUY2 format on the screens 106 and 206, theimages shown on the screens 106 and 206 have different luminance andchrominance. Because the color converter 128 of the Direct Show module124 and the color converter 228 of the Direct Show module 224 usedifferent conversion coefficients to convert the image signals S_(YUV)and S_(YUY2) of different formats to the image signals S_(RGB) of theRGB format, the image signals S_(RGB) output by the color converters 128and 228 have different luminance and chrominance. The color converter128 converts the image signal S_(YUV) to the image signal S_(RGB)according to the following algorithm:

$\begin{matrix}{{\begin{bmatrix}R \\G \\B\end{bmatrix} = {\begin{bmatrix}1 & 0 & 1.402 \\1 & {- 0.344} & {- 0.715} \\1 & 1.77 & 0\end{bmatrix}\begin{bmatrix}Y \\{U - 128} \\{V - 128}\end{bmatrix}}},} & (1)\end{matrix}$

wherein Y, U, and V are respectively luma components, first chromacomponents, and second chroma components of the image signal S_(YUV),and R, G, and B are respectively intensities of a red primary color, agreen primary color, and a blue primary color of the image signalS_(RGB). On the contrary, the color converter 228 converts the imagesignal S_(YUY2) to the image signal S_(RGB) according to the followingalgorithm:

$\begin{matrix}{{\begin{bmatrix}R \\G \\B\end{bmatrix} - {\begin{bmatrix}1.164 & 0 & 1.598 \\1.164 & {- 0.391} & {- 0.813} \\1.164 & 2.016 & 0\end{bmatrix}\begin{bmatrix}{Y - 16} \\{U - 128} \\{V - 128}\end{bmatrix}}},} & (2)\end{matrix}$

wherein Y, U, and V are respectively the luma components, the firstchroma components, and the second chroma components of the image signalS_(YUY2), and R, G, and B are respectively intensities of a red primarycolor, a green primary color, and a blue primary color of the imagesignal S_(RGB). According to the algorithms (1) and (2), the conversioncoefficients of the algorithm (2) are greater than the conversioncoefficients of the algorithm (2). Thus, the image signal S_(RGB)converted from the image signal S_(YUY2) of a YUY2 format has a higherintensity than that of the image signal S_(RGB) converted from the imagesignal S_(YUV) of a YUV format.

To lower the intensity of the image signal S_(RGB) converted from theimage signal S_(YUY2) of a YUY2 format, the invention provides a cameradevice which can adjust the luminance and chrominance of the imagesignal S_(YUY2) of a YUY2 format before the image signal S_(YUY2) isoutput. Referring to FIG. 4, a block diagram of a camera device 402outputting an image signal of a YUY2 format and a host 404 receiving theimage signal according to the invention is shown. Referring to FIG. 5, aflowchart of a method 500 for processing image signals of the cameradevice 402 and the host 404 shown in FIG. 4 according to the inventionis shown. In one embodiment, the camera device 402 comprises a sensor412 and a controller 414, and the controller 414 comprises an imageprocessor 416 and a subsequent processor 418. The host 404 is identicalto the host 204 shown in FIG. 2 and comprises a USB video class driver422 and a Direct Show module 424.

First, the sensor 412 detects an image to generate a first image signalS_(RGB) of a RGB format (step 502). The image processor 414 of thecontroller 414 then converts the first image signal S_(RGB) to a secondimage signal S_(YUY2) with a YUY2 format (step 504). The subsequentprocessor 418 then adjusts luma components and chroma components of thesecond image signal S_(YUY2) generated by the image processor 416 toobtain a third image signal S_(YUY2′). First, the subsequent processor418 adjusts a plurality of luma components of the second image signalS_(YUY2) according to a first function to obtain a plurality of adjustedluma components of the third image signal S_(YUY2′) (step 506). Thesubsequent processor 418 then adjusts a plurality of first chromacomponents of the second image signal S_(YUY2) according to a secondfunction to obtain a plurality of first adjusted chroma components ofthe third image signal S_(YUY2′) (step 508). The subsequent processor418 then adjusts a plurality of second chroma components of the secondimage signal S_(YUY2) according to a third function to obtain aplurality of second adjusted chroma components of the third image signalS_(YUY2′) (step 510). In one embodiment, the first function is a linearfunction with variables of the luma components, the second function is alinear function with variables of the first chroma components, and thethird function is a linear function with variables of the second chromacomponents. In one embodiment, the subsequent processor 418 adjusts theluma components and the chroma components of the second image signalS_(YUY2) according to the following algorithm:

$\begin{matrix}{{\begin{bmatrix}Y^{\prime} \\U^{\prime} \\V^{\prime}\end{bmatrix} = {{\begin{bmatrix}0.859 & 0 & 0 \\0 & 0.875 & 0 \\0 & 0 & 0.875\end{bmatrix}\begin{bmatrix}{Y + 16} \\{U - 128} \\{V - 128}\end{bmatrix}} + \begin{bmatrix}0 \\128 \\128\end{bmatrix}}},} & (3)\end{matrix}$

wherein Y, U, and V are respectively the luma components, the firstchroma components, and the second chroma components of the second imagesignal S_(YUY2), and Y′, U′, and V′ are respectively the adjusted lumacomponents, the first adjusted chroma components, and the secondadjusted chroma components of the third image signal S_(YUY2′).

The camera device 402 then transmits the third image signal S_(YUY2′) tothe host 404 (step 512). In one embodiment, a USB video class driver 422of the host 404 drives a universal serial bus (USB), and the third imagesignal S_(YUY2′) is transmitted via the universal serial bus. The DirectShow module 424 of the host 404 then converts the third image signalS_(YUY2′) with a YUY2 format to a fourth image signal S_(RGB) with a RGBformat (step 514). Because the subsequent processor 418 has lowered theluminance and chrominance of the second image signal S_(YUY2′) accordingto the algorithm (3), the color converter 428 can now convert the thirdimage signal S_(YUY2′) according to the algorithm (2) to obtain a fourthimage signal S_(RGB) comprising a red primary color, a green primarycolor, and a blue primary color with a lower intensity. The host 404then transmits the fourth image signal S_(RGB) to a screen 406 (step515), and the screen 406 displays the fourth image signal S_(RGB) (step516). Because the subsequent processor 418 has adjusted the luminanceand chrominance of the third image signal S_(YUY2′) output by the cameradevice 402, an image displayed on the screen 406 has the same luminanceand chrominance with an image derived from an image signal S_(MJPG) withthe MJPG format shown in FIG. 1.

While the invention has been described by way of example and in terms ofpreferred embodiment, it is to be understood that the invention is notlimited thereto. To the contrary, it is intended to cover variousmodifications and similar arrangements (as would be apparent to thoseskilled in the art). Therefore, the scope of the appended claims shouldbe accorded the broadest interpretation so as to encompass all suchmodifications and similar arrangements.

1. A camera device, coupled to a host, comprising: a sensor, detectingan image to generate a first image signal with an RGB format; and acontroller, comprising: an image processor, converting the first imagesignal to a second image signal with a YUY2 format; and a subsequentprocessor, adjusting a plurality of luma components, a plurality offirst chroma components, and a plurality of second chroma components ofthe second image signal to obtain a plurality of adjusted lumacomponents, a plurality of first adjusted chroma components, and aplurality of second adjusted chroma components of a third image signal,wherein the host receives the third image signal output by the camera,uses a Direct Show module to convert the third image signal to a fourthimage signal with an RGB format, and outputs the fourth image signal toa screen.
 2. The camera as claimed in claim 1, wherein the host receivesa fifth image signal with a motion joint photographic experts group(MJPG) format, decodes the fifth image signal to obtain a sixth imagesignal with a YUV format, and uses the Direct Show module to convert thesixth image signal to a seventh image signal with an RGB format, whereinadjustment of the subsequent processor makes the luminance andchrominance of the fourth image signal equal to that of the seventhimage signal.
 3. The camera as claimed in claim 1, wherein thesubsequent processor adjusts the luma components of the second imagesignal according to a first function to obtain the adjusted lumacomponents of the third image signal, adjusts the first chromacomponents of the second image signal according to a second function toobtain the first adjusted chroma components of the third image signal,and adjusts the second chroma components of the second image signalaccording to a third function to obtain the second adjusted chromacomponents of the third image signal.
 4. The camera as claimed in claim3, wherein the first function is a linear function with variables of theluma components, the second function is a linear function with variablesof the first chroma components, and the third function is a linearfunction with variables of the second chroma components.
 5. The cameraas claimed in claim 1, wherein the subsequent processor adjusts thesecond image signal to obtain the third image signal according to thefollowing algorithm: ${\begin{bmatrix}Y^{\prime} \\U^{\prime} \\V^{\prime}\end{bmatrix} = {{\begin{bmatrix}0.859 & 0 & 0 \\0 & 0.875 & 0 \\0 & 0 & 0.875\end{bmatrix}\begin{bmatrix}{Y + 16} \\{U - 128} \\{V - 128}\end{bmatrix}} + \begin{bmatrix}0 \\128 \\128\end{bmatrix}}},$ wherein Y, U, and V are respectively the lumacomponents, the first chroma components, and the second chromacomponents of the second? image signal, and Y′, U′, and V′ arerespectively the adjusted luma components, the first adjusted chromacomponents, and the second adjusted chroma components of the third imagesignal.
 6. The camera as claimed in claim 1, wherein the Direct Showmodule converts the third image signal to the fourth image signalaccording to the following algorithm: ${\begin{bmatrix}R \\G \\B\end{bmatrix} - {\begin{bmatrix}1.164 & 0 & 1.598 \\1.164 & {- 0.391} & {- 0.813} \\1.164 & 2.016 & 0\end{bmatrix}\begin{bmatrix}{Y - 16} \\{U - 128} \\{V - 128}\end{bmatrix}}},$ wherein Y, U, and V are respectively the adjusted lumacomponents, the first adjusted chroma components, and the secondadjusted chroma components of the third image signal, and R, G, and Bare respectively intensities of a red primary color, a green primarycolor, and a blue primary color of the fourth image signal.
 7. Thecamera as claimed in claim 2, wherein the Direct Show module convertsthe sixth image signal to the seventh image signal according to thefollowing algorithm: ${\begin{bmatrix}R \\G \\B\end{bmatrix} = {\begin{bmatrix}1 & 0 & 1.402 \\1 & {- 0.344} & {- 0.715} \\1 & 1.77 & 0\end{bmatrix}\begin{bmatrix}Y \\{U - 128} \\{V - 128}\end{bmatrix}}},$ wherein Y, U, and V are respectively a plurality ofluma components, a plurality of first chroma components, and a pluralityof second chroma components of the sixth image signal, and R, G, and Bare respectively intensities of a red primary color, a green primarycolor, and a blue primary color of the seventh image signal.
 8. Thecamera as claimed in claim 1, wherein a universal serial bus (USB) iscoupled between the camera and the host, and the third image signal istransmitted from the camera to the host via the universal serial bus. 9.An image processing method, wherein a camera device is coupled to ahost, comprising: detecting, by a sensor an image, to generate a firstimage signal with an RGB format; converting, by an image processor, thefirst image signal to a second image signal with a YUY2 format;adjusting, by a subsequent processor, a plurality of luma components, aplurality of first chroma components, and a plurality of second chromacomponents of the second image signal to obtain a plurality of adjustedluma components, a plurality of first adjusted chroma components, and aplurality of second adjusted chroma components of a third image signal;and transmitting the third image signal to the host, wherein the hostreceives the third image signal output by the camera, uses a Direct Showmodule to convert the third image signal to a fourth image signal withan RGB format, and outputs the fourth image signal to a screen.
 10. Theimage processing method as claimed in claim 9, wherein the host receivesa fifth image signal with a motion joint photographic experts group(MJPG) format, decodes the fifth image signal to obtain a sixth imagesignal with a YUV format, and uses the Direct Show module to convert thesixth image signal to a seventh image signal with an RGB format, whereinadjustment of the subsequent processor makes the luminance andchrominance of the fourth image signal equal to that of the seventhimage signal.
 11. The image processing method as claimed in claim 9,wherein adjustment of the second image signal comprises: adjusting, bythe subsequent processor, the luma components of the second image signalaccording to a first function to obtain the adjusted luma components ofthe third image signal; adjusting, by the subsequent processor, thefirst chroma components of the second image signal according to a secondfunction to obtain the first adjusted chroma components of the thirdimage signal; and adjusting, by the subsequent processor, the secondchroma components of the second image signal according to a thirdfunction to obtain the second adjusted chroma components of the thirdimage signal.
 12. The image processing method as claimed in claim 11,wherein the first function is a linear function with variables of theluma components, the second function is a linear function with variablesof the first chroma components, and the third function is a linearfunction with variables of the second chroma components.
 13. The imageprocessing method as claimed in claim 9, wherein the second image signalis adjusted to obtain the third image signal according to the followingalgorithm: ${\begin{bmatrix}Y^{\prime} \\U^{\prime} \\V^{\prime}\end{bmatrix} = {{\begin{bmatrix}0.859 & 0 & 0 \\0 & 0.875 & 0 \\0 & 0 & 0.875\end{bmatrix}\begin{bmatrix}{Y + 16} \\{U - 128} \\{V - 128}\end{bmatrix}} + \begin{bmatrix}0 \\128 \\128\end{bmatrix}}},$ wherein Y, U, and V are respectively the lumacomponents, the first chroma components, and the second chromacomponents of the second? image signal, and Y′, U′, and V′ arerespectively the adjusted luma components, the first adjusted chromacomponents, and the second adjusted chroma components of the third imagesignal.
 14. The image processing method as claimed in claim 9, whereinthe Direct Show module converts the third image signal to the fourthimage signal according to the following algorithm: ${\begin{bmatrix}R \\G \\B\end{bmatrix} - {\begin{bmatrix}1.164 & 0 & 1.598 \\1.164 & {- 0.391} & {- 0.813} \\1.164 & 2.016 & 0\end{bmatrix}\begin{bmatrix}{Y - 16} \\{U - 128} \\{V - 128}\end{bmatrix}}},$ wherein Y, U, and V are respectively the adjusted lumacomponents, the first adjusted chroma components, and the secondadjusted chroma components of the third image signal, and R, G, and Bare respectively intensities of a red primary color, a green primarycolor, and a blue primary color of the fourth image signal.
 15. Theimage processing method as claimed in claim 10, wherein the Direct Showmodule converts the sixth image signal to the seventh image signalaccording to the following algorithm: ${\begin{bmatrix}R \\G \\B\end{bmatrix} = {\begin{bmatrix}1 & 0 & 1.402 \\1 & {- 0.344} & {- 0.715} \\1 & 1.77 & 0\end{bmatrix}\begin{bmatrix}Y \\{U - 128} \\{V - 128}\end{bmatrix}}},$ wherein Y, U, and V are respectively a plurality ofluma components, a plurality of first chroma components, and a pluralityof second chroma components of the sixth image signal, and R, G, and Bare respectively intensities of a red primary color, a green primarycolor, and a blue primary color of the seventh image signal.
 16. Theimage processing method as claimed in claim 9, wherein a universalserial bus (USB) is coupled between the camera and the host, and thethird image signal is transmitted from the camera to the host via theuniversal serial bus.